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Abstract. We prove the Jarzynski relation for general stochastic processes including 
non-Markovian systems with memory. The only requirement for our proof is the 
existence of a stationary state, therefore excluding non-ergodic systems. We then 
show how the concepts of stochastic thermodynamics can be used to prove further 
exact non-equilibrium relations like the Crooks relation and the fluctuation theorem 
on entropy production for non-Markovian dynamics. 
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1. Introduction 

The Jarzynski relation pQ connects non-equilibrium work values W spent in driving a 
system initially in equilibrium with the change of free energy AF = Fb — Fa between 
initial (A) and final state (B) through the nonlinear average 

(e-W) = e -^. (1) 

Here, (3 = (k^T)" 1 is the inverse temperature of the heat bath the system is coupled 
to and &b is Boltzmann's constant. The Jarzynski relation has found wide-spread 
application in both experiments and computer simulations (for reviews, see [2J [3]). 
While ([1]) was derived originally for deterministic dynamics [1] and then generalized 
to stochastic Markov processes pE], its extension to general non-Markovian dynamics is 
still open. The special case of a driven harmonic oscillator has been treated analytically 
and numerically in [5]. In [6], the case of non-equilibrium baths with memory is discussed 
for the entropy production. 

In this paper, we will first give a general proof of the Jarzynski relation which 
basically extends a previous proof for Markovian dynamics [7]. We then specialize 
to Gaussian noise and discuss the role of time reversal. This will allow us to show 
that further non-equilibrium relations like the Crooks relation 0, E] and the detailed 
fluctuation theorem in non-equilibrium steady states [TQl [TTj, [12] hold as well. Together 
with their counterparts holding for deterministic thermostated dynamics [131 El f!5] . 
all these non-equilibrium relations show a surprising robustness against the underlying 
dynamics. 

2. Proof of the Jarzynski relation for general stochastic processes 

Let the energy of the system be given by a Hamiltonian H(T, A), where T is a point in 
phase space. We assume that we can control the system externally through a change 
of the parameter A where the function A(r) is called the protocol. We will consider 
trajectories T(r) in the interval 

t ^ ^ r ^ t < tx (2) 

involving four times. At the lower boundary to we prepare the system such that it 
retains no memory of earlier times. We then observe single trajectories from i to t\. 
However, we will drive the system through a change of A only during the inner interval 
^ t ^ t such that A is constant outside. The change of energy identified as the work 
spent along a single trajectory T(r) is then 

drA(r) — (r(r),A(r)). (3) 

Hence, the work depends only on the inner section of the trajectory, which is the first 
ingredient for the proof. In the following, we will drop the implicit dependence on the 
protocol in the argument of functionals. 
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The second ingredient for the proof is the time evolution equation 

d T p(T,T) = L(r;t Q )p(T,T) (4) 

of the distribution p(T, r) determining the probability to find the system in a specific 
region of phase space. The evolution of this distribution is governed by the operator 
L(r;t ). In the Markov case, the operator L(r;t ) — > L m (X(r)) is the generator of a 
semi-group [IT]. It then completely defines the stochastic process. It is independent of 
to and therefore it does not depend on the details of preparation while it depends on 
time due to the change of the external parameter A. 

It is somewhat surprising that the same, apparently time-local, equation holds 
also for non-Markovian processes [THl HZj- This can be understood by realizing that 
the complete information about processes with memory is contained in the transition 
probability depending on the whole history rather than in the single-point distribution 
p(T,r). We denote with U(t'\t; to) the operator that propagates the system from time 
r < t' to the later time r' . The propagator actually depends on the whole function 
A(t) up to t' since any change of the protocol will have consequences for the following 
evolution. From the propagator, we can define the operator 

L s (r;t )=d T/ U(r'\r;t ) (5) 

t'=t+ 

describing a "substitute", non-stationary Markov process, L(r;io) —* L a (r',to), which 
leads to the same single-point distribution p(T, r) but to a different transition probability 
than the non-Markovian process [16J. In particular, knowledge of the operator (J5j) is 
not sufficient to calculate correlation functions. In contrast to the Markov case, the 
dependence on the control parameter A of the operator (jSJ) is implicit. In the appendix, 
we give an explicit example for such a substitute operator. 

We restrict our proof to dynamics with a unique steady state, i.e., for fixed A the 
system will relax towards a unique probability distribution p s (T, A) depending on the 
control parameter, lim T ^ QO p(r, r) — ► p s (T, A). This is equivalent to ergodic processes 
with or without memory (see Ref. [18] for a discussion of non-Markovian processes which 
break ergodicity). In the absence of non-conservative driving, the stationary distribution 
must be the equilibrium Gibbs-Boltzmann distribution 

p eq (r,A) = [Z(A)]- 1 e-^ r ' A ), (6) 

where the partition function 

Z(X) = [ dr e-? H ^ (7) 



determines the free energy F(X) = — j3~ l In Z(X). Then Za,b is the partition function 
of the initial and the final state, respectively. 

The third ingredient to the proof is the property that the equilibrium distribution 
is the stationary solution 

L(r;to)p cq (T,X(r)) = (8) 
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for the corresponding value A = A(r) of the control parameter. Whereas this is evident 
in the case of a Markovian operator, due to the implicit dependence on A it is not so 
obvious in the non-Markovian case and we give a proof by contradiction. First we note 
that for a proper Markovian substitute process, the operator (jSJ) must have a stationary- 
solution. Now suppose that at time r' we stop the process and hold the parameter fixed 
with value A = A(r'). Under very general conditions, which are fulfilled by any transition 
probability, the Perron- Frobenius theorem ensures that the propagator U (t|t'; to) has an 
eigenstate pi(T; r, r') corresponding to the eigenvalue 1 depending on r' and in principle 
also depending on r, i.e., 

U(T\T';t )p 1 (T,r')=p 1 (r,r'). (9) 

Furthermore, this eigenstate pi(T; r, r') is a normalized, non-negative probability 
distribution. From the definition ([5]), we calculate 

L s (r'; t ) Pl (r, r') = lim - \ll(r' + e\r'; t ) Pl (r, r') - Pl (r, r')] ^ (10) 

which is non-zero for both arbitrary functions p\ and for the eigenfunction p\ (r, r') of the 
propagator if the latter depends on r since r does not match the leading time argument 
of the propagator. This would mean that the substitute operator (jSJ) has no stationary 
solution. This contradiction is resolved only if the eigenfunction p\ (r') is independent of 
t. Moreover, taking then the limit r — > oo in (jUJ), we find from the ergodicity condition 
that pi(T,r') = p eq (r, A). Finally, we note that due to causality, we do not have to 
actually stop the process at a r' since the system cannot depend on the future protocol 
and (jSJ) must hold for all times r. 

With these three ingredients, the proof of the Jarzynski relation ([I]) becomes simple. 
We prepare the system at time i ^ in equilibrium and start to drive the system 
at r = until r = t following the protocol A(r). Inspecting the expression for the 
work (j3J), we see that its instantaneous change H(T,r) = Xd\H(T, A) only depends on 
the actual state T the system is in. Hence, the operator L(r; t ) is all we need to prove 
the Jarzynski relation including non-Markovian processes. To this end, we consider the 
joint probability p(T,w,r) for finding the system in state T at time r and for having 
accumulated an amount of work w up to this time [3, [19] . A change of the state or the 
control parameter A will lead to a probability current j w = Hp in the direction of the 
w coordinate, hence the equation of motion for the joint probability becomes 

d T p(T, w, r) = [l(t; t ) - H(T, r)9 w ] p{T, w, r) (11) 

due to the conservation of probability. We can prove the Jarzynski relation (CQ) by first 
defining the function 

r+oo 

V>(r,r)=/ dw p{T,w,T)e-^ w . (12) 



Since the probability for extreme work values w — ► ±oo vanishes, after one integration 
by parts the equation of motion becomes 

^(r, r) = \1{t- t ) - PH(T, r)l ^(r, r). (13) 
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A solution of this equation is the equilibrium Boltzmann factor 

^{Y,r) = Z^e-^ T ^ (14) 

provided L(r; to)^(T, r) = holds for all r as discussed above, see (JED- The solution ( fl4l) 
obeys both the initial equilibrium condition ip(T,0) = peq(r,A(0)) and f dT ^(T,t) = 
Zb/Za which implies the Jarzynski relation ([T|). 

We have thus proved that the Jarzynski relation (CQ) holds for any kind of non- 
Markovian noise by exploiting the existence of a time-local substitute operator (j4]) which 
annihilates the Gibbs-Boltzmann distribution p e q(T, A) for any A reached during the 
process. This proof only involves the work definition ([3]) but no other thermodynamic 
notions like heat and entropy, which we will discuss now. 



3. Stochastic thermodynamics 

The Jarzynski relation (CQ) can be embedded into the larger framework of stochastic 
thermodynamics. The crucial idea is to extent the notion of work and heat to small, 
stochastic systems coupled to a heat bath in the following way: one first identifies the 
energy change caused externally as the work and then the heat dissipated due to the 
interaction with the bath follows from the first law [20] . The first law thus reads 

Q\T(t)]=W\T(t)]-AH (15) 

with the change of internal energy AH = H(T(ti), A(t)) — H(T{t ), A(0)) along the 
specific trajectory T(r). The sign of the heat is convention, here we choose it to be 
positive if energy is dissipated into the bath. Using the definition of the work (J3j), we 
write the right hand side of equation ffl5|) under one integral sign. By inserting the total 
derivative of the energy, the heat becomes the functional 

Q\F(r)] = - jT dr f (r) ■ ^(r(r), A(r)). (16) 

So far, we did not make any assumptions about the bath and the dynamics of the 
system and hence the expressions for work and heat should hold for both Markovian 
and non-Markovian dynamics. 



4. Time reversal 



The identification of the heat allows for a second route to derive the Jarzynski relation (TjQ) 
via time reversal [9], [121 [21] ■ For notational simplicity, we restrict our discussion to one 
overdamped degree of freedom T = x moving in the potential H = V(x, A). We define 
the functional 

R\x(t)\ = In P[X{T)] = In p [*fr)l*oW*o) (17) 

which fulfills the relation (exp[— R]) = 1 by definition since P[x(r)] is the probability of 
a trajectory x{r). In the second step, we have separated the initial state x = x(t ) from 
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the conditional probability P[x(r)|x ] times the probability distribution of the initial 
state. The path x(r) = x(t + 1\ — r) denotes the time reversal starting in X\ = x(ti). 

We model the dynamics of a system coupled to a non-Mar kovian bath with Gaussian 
noise rj through the generalized Langevin equation 

7 (t - r) o ±(t) = -V'(x(t), X(t)) + V (t) (18) 

with friction kernel 7(1"). The prime denotes derivation with respect to x. For 
convenience, we define the operation 

g(r) o h(r) = [ * dr g{r)h{r) (19) 



Jt 

where integration is carried out over the free variable r analogous to Einstein's sum 
convention. If one or both of the functions depend on x then by this short notation we 
mean g{r) = g(x(r), A(r)). 

Equation ( Tl8i) includes the Markov case through choosing a time-local friction kernel 
7(r) = 27<5(t) with friction coefficient 7. The bath correlation function is defined as 

C(n - r 2 ) = (77(71)77(72)) (20) 

and the system is guaranteed to equilibrate with the heat reservoir through Kubo's 
second fluctuation-dissipation theorem [22] 

1VH = I (21) 

[0 T < 0. 

The probability of a certain noise history r](r) obeys P[t](t)] > for any continuous path 
77 (r) and normalization j[dr](r)] P[t](t)] = 1 with functional measure [dr/(r)]. Gaussian 
noise is completely defined by its (zero) mean and correlations C(r). We therefore write 
P[r](r)} = exp{— A[?7(r)]} introducing the quadratic "action" functional 

A[v(r)) = \-n{n) o K{ Tl - r 2 ) o v (t 2 ). (22) 

The symmetric noise kernel K(t) = K(—t) is the operator inverse of the bath correlation 
function C(r), 

C(n - r) o K(r - r 2 ) = 5(n - r 2 ). (23) 

We change variables from rj to x with the probability of a single trajectory 

P[x(r)\x } = J[x{t)} exp {-A s [x{r)\ - A a [x(r)]}. (24) 

This change of variables makes it necessary to consider the conditional probability since 
the noise history does not determine the initial state Xq. The total action becomes a 
sum of two terms defined as 

A s [x{r)} = ^ / (r 1 )oir(r 1 -r 2 )oF / (r 2 ) + [ 7 (r 1 - r) o x (r)] otf (ti-t 2 )o [7 (r 2 - r) o i(r)] (25) 
and 

A a [x(r)} = bin - r) o x{t)\ o - r 2 ) o V'(r 2 ), (26) 



Stochastic Thermodynamics for Non-Markovian Processes 



7 



where we have replaced the noise through the generalized Langevin equation fll8p . This 
change of variables involves further the Jacobian J[a;(r)] = det[5i](t) / 5x(t)] given by 
the functional determinant. For time reversal, we run along the trajectory x(t) in the 
opposite direction from ti to t . In the integrals, this amounts both to the substitution 
x i — > —x and to inverting the time argument of the kernels. Under these operations, 
both the symmetric action A s and the Jacobian J stay invariant. The antisymmetric 
action of the time-reversed trajectory becomes 

MHt)] = - [7(t - n) o ±(t)] o K( n - r 2 ) o V'(t 2 ) (27) 

and hence the sum is 

- AMr)] + A a [x(r)} = -x(n) o M{r x - r 2 ) o V'(r 2 ) (28) 

with kernel 

M(n - r 2 ) = [ 7 (n - r) + 7 (n - r)] o K(t - r 2 ). (29) 

The sum in the square brackets equals the symmetric bath correlation function C(ti — t) 
through use of the fluctuation-dissipation theorem ( l2~Tj) . Following ( l23l) . the kernel then 
reduces to the 5-function. The functional R finally reads 

-R[x{t)\ = Q[x(t)} +AV-AF = W[x(t)} - AF (30) 

with the heat ffl6|) independent of the actual bath correlation function C(r). The only 
requirement is that the bath itself is and stays in equilibrium as it is in the case of 
Markov processes. 

5. Discussion and the path to proving further non-equilibrium relations 

We can now discuss the role of the two times to and t\. In equilibrium, fluctuations 
have no memory. Since the trajectory x(t) is just the mirror image of x(r), their 
probabilities must then be equal, P[x(r)] = P[x(t)], with R — 1. We can therefore 
chose an arbitrary interval to ^ r ^ t\ during which we observe the trajectory and the 
heat fulfills Q = — AV^. When we now drive the system through the manipulation of 
A, we can actually choose the driving interval as the observation interval, to —>■ 0~ and 
ti —>■ t + . However, the noise kernel K(t) defined through fT25|) then depends on the 
two times to and t%. This reflects the fact that both the forward and the time-reversed 
trajectory are cut off although the force at the boundary still remembers the velocity of 
earlier times. 

Although we have shown the relation fl30|) only for Gaussian noise, the fact that 
in the first part of the paper we have proven the Jarzynski relation without assuming 
a specific type of noise suggests that the relation (130]) is also valid more generally. 
However, a direct treatment of non-Gaussian noise within the path integral formalism 
seems technically challenging. 



Stochastic Thermodynamics for Non-Markovian Processes 



8 



The fact that (1301) holds also for non-Markovian dynamics implies the validity of 
other non-equilibrium relations. First, from the definition (1TT|) . one can derive the 
Crooks relation |9J 

Pr(-W) _ m -AF) r oiX 

if we distinguish between the probability distribution of the work Pf,r(W) spent in the 
forward and time-reversed processes, respectively. Second, for an equilibrated bath, we 
can still identify the dissipated heat as entropy change in the heat bath, As m = (3Q. 
With the change of entropy of the system, 

As = sih) - s(t ), s(t) ee -lnp(r(r),r), (32) 

the fluctuation theorem for the total entropy production As tot = As m + As [12] 

( e - Astot ) = 1 (33) 

remains valid. 

Finally, our analysis of non-Markovian processes can be easily extended to systems 
driven by nonconservative forces which for constant A reach a non-equilibrium steady 
state with probability distribution p s (T). In this case, we have to include the 
nonconservative forces / in the external work, leading to 



W[T(t)] ee f dr 
Jo 



A(T)^(T(T),A(T)) + /(T)-f(T) 



(34) 



The heat is still determined through the first law (ITST) . If we generalize the functional 
R from equation ( 1TT1) by replacing the equilibrium distribution p eq with the stationary 
distribution p s , it is straightforward to derive the relation 

i?[r(r)]=/3g[r(r)] + ln^M, (35) 

from which the integral fluctuation theorem for entropy production (I33p follows. 
Moreover, in the case of stationary driving (A = 0) also the detailed fluctuation 
theorem PH EH E2] 

follows for a finite time interval, where P(As t ot) is the probability distribution of the 
total entropy production. The relation (J55|) is also found in deterministic steady state 
systems in the long-time limit [T3J [HI [15] . 



6. Summary and outlook 

As our main result, we have shown that the Jarzynski relation holds for general ergodic 
systems governed by stochastic dynamics including non-Markovian processes. We 
have further confirmed in the case of Gaussian noise that the relations (|30|) and (1331) 
between the heat Q and the functional R, which serves as convenient starting point to 
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derive further exact non-equilibrium relations, still holds for non-Markovian processes. 
Therefore this class of exact non-equilibrium relations, of which the Jarzynski relation 
is arguably the most prominent, shows a surprising robustness against the underlying 
dynamics. An open question which will require further investigation is to which extent 
the concepts discussed in this paper can be generalized to non-ergodic systems. 



Appendix A. Substitute operator for a moving trap 



As an illustration, we calculate the substitute operator in case of a particle moving 
in one dimension with position x which is trapped in a harmonic potential V(x, A) = 
(k/2)(x — A) 2 . The generalized Langevin equation (118p then becomes linear and can be 
solved by Laplace transformation as 

x(t) = G 1 (t)x + [ dr G 2 {t - r)[kX(r) + t)(t)], 



where the two kernels are given as the inverse Laplace transform of Gi(s) = A f(s)G2(s) 
and G2(s) = [57(5) + respectively. The system is prepared at time t = in 

equilibrium with initial position xq drawn from p eq (x, 0). Due to the change of the 
external parameter A, the mean 

m{t) = (x(t)) = [ dr kG 2 (t - r)A(r) 



is a functional of A(r). Without loss of generality, we have set A(0) = and hence 
(x ) = 0. 

The substitute operator for one-dimensional Gaussian processes has been worked 
out explicitly in [16] reading in general 



X(t)x + fi(t)--a(t)d x 



L s (t) = -d x 

The functions fi(t) and a(t) are determined through the differential equations 

m(t) = fi(t) + x(t)m(t), v(t) = &{t) + 2 X (t)v(t) 

with time-dependent mean m(t) and variance v(t). The correlation function 

([x(t) - m(t)][x(t') -m(t')\) 



x(t,t') 



v(t>) 



(a.i; 



with x(t,t) — 1 determines = d r xij,t)\ T=t . 

To be more specific, we choose an exponential friction kernel 



7(t) = Ke 



-Kt 



G 2 (t) = (R/k) 2 e- Kt + 



S(t) 



S + K ' K + k 

with inverse time scale R = K,k/(n + k). In the Markov limit, k — > 00 yields R — * k as 
expected. Using the explicit expression for the kernel we calculate the mean 



m 



(r) = e- R{T - T,) m(T') + A [l - e'^-^ 
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where we have stopped the process at r' with parameter A = A(r'). This equation 
shows the basic features of ergodic non-Markovian processes. For fixed A, the mean 
m(r — > oo) — > A relaxes towards this value. It is a functional of A(r) up to r' and 
afterwards depends on the time difference r — r' only. The time derivative yields 
m(r) = —Rm{r)+R\ and indeed a straightforward calculation of 11A.1I) confirms \ — 
Therefore, we have fi = R\ and since we do not change the strength of the trap, the 
variance is v — l/(/3k) leading to & — 2R/(f3k). Hence, the substitute operator for fixed 
A becomes 

1 



k8 x 



(x-A) + -a, 



with stationary solution p e q(x, A) oc exp[— (3{k/2)(x — A) 2 ] for all times r ^ r'. 
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